Bit-rate control method and device combined with rate-distortion optimization

ABSTRACT

In the invention, a rate distortion optimization (RDO) based rate control scheme is comprised of following two steps: first, does bit allocation for every frame in a GOP, and based on the allocated bits, a predicted quantization parameter is used to do the first rate distortion optimization mode selection for every macroblock in the current frame; second, the information of the current macroblock collected from the first rate distortion mode selection is used to calculate a final quantization parameter for rate control, and if the final quantization parameter is different from the predicted one, a second rate distortion mode selection will be executed again. A rate distortion optimization based rate control implementation includes following modules: a video coding encoder module (for example, H.264/JVT processing module), rate distortion optimization based macroblock mode selection and adaptive quantization module, virtual buffer, and global complexity estimation module. As RDO and rate control are considered together in the invention, the RDO based rate control scheme can achieve better coding performance while with accurate target bitrate control.

BACKGROUND OF THE INVENTION

Advanced video coding techniques are important for multimedia storageand transmission. For this reason, many video coding standards have beenstandardized. H.264 is the latest video coding standard. H.264/AVCstandard jointly developed by ISO and ITU-T—Joint Video Team (JVT), alsoknown as MPEG-4 Part 10 and H.264 in the H.26x serial standards, hassubstantially outperformed the previous video coding standards byutilizing a variety of temporal and spatial predictions. Rate control isan important technique although it does not belong to the normative partin video coding standards. However, without rate control any videocoding scheme would be practically useless in many applications becausethe client buffer may often under-flow and over-flow when a channel usedto deliver the compressed stream is of constant bandwidth. Therefore,every video coding standard has its own rate control technique, forexample, TM5 for MPEG-2 and TMN8 for H.263.

RDO is one of important video coding techniques. It is used to selectoptimal motion vectors an optimal coding mode for every macroblock. Yetthe RDO used in H.264 test model makes it difficult to adopt theexisting rate control techniques. Because rate control usually requiresa pre-determined set of motion vectors and coding modes to select thequantization parameter, whereas RDO requires a pre-determinedquantization parameter to select motion vectors and coding modes. On theother hand, as the complexity ratio between coded frame, the bitallocation model and adaptive quantization scheme should also beimproved. The invention is a method and apparatus for rate distortionoptimization based rate control. The invention can be used for videostreaming, transmission, and storage coding.

SUMMARY OF THE INVENTION

The invention is to provide a method and apparatus of rate control for avideo encoder, in which rate distortion optimization technique is usedto improve coding efficiency.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is the flow chart of an embodiment according to the presentinvention.

FIG. 2 is the schematic structural view of the apparatus according tothe present invention.

As shown in FIG. 2, a rate distortion optimization based rate controlimplementation includes following modules: JVT processing module, ratedistortion optimization based macroblock mode selection module, virtualbuffer, and global complexity estimation module.

JVT processing module receives the input frame data, and it is connectedwith RDO mode selection module, virtual buffer module and globalcomplexity estimation module;

RDO mode selection module is connected with virtual buffer and globalcomplexity estimation module. It receives the input signal from JVTprocessing module, and processes it based on the virtual buffer moduleand global complexity module status. In the last, the output signal issent back to JVT processing module, JVT module will output the finalcoded macroblock.

Before coding a GOP, does bit allocation for the pictures in the GOPwith the average picture size; The average picture size is calculatedas:

R/F=R÷F, here, R is the target bit rate. F is the picture rate. R/F isthe average picture size.

The bit allocation adjustment in the coded GOP is shown as follows:

$\begin{matrix}{T_{b} = {\max\;\left\{ {\frac{R}{N_{b} + \frac{N_{p}K_{b}{Xp}}{K_{p}X_{b}}},\frac{{bit}\text{\_}{rate}}{8 \times {picture\_ rate}}} \right\}}} \\{T_{p} = {\max\;\left\{ {\frac{R}{N_{p} + \frac{N_{b}K_{p}X_{b}}{K_{b}X_{p}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} \\{T_{i} = {\max\;\left\{ {\frac{R}{1 + \frac{N_{p}X_{p}}{K_{p}X_{i}} + \frac{N_{b}X_{b}}{K_{b}X_{i}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}}\end{matrix}$

here, T_(I), T_(p) and T_(b) is the bits allocated to the I, P or Bframe respectively. N_(i), N_(p) and N_(b) is the remained none coded I,P or B frames in the GOP respectively. X_(i), X_(p) and X_(b) is theglobal complexity estimation for the I, P or B frame respectively and isdefined as the multiplier between coded bits and average quantizationparameter for the frame.

bit_rate is the target bit rate. picture rate is the frame rate.

K_(p) and K_(b) are constants. K_(p), K_(b) means the complexity rationbetween P, B frame and I frame respectively.

R is the remained bits for the GOP, and after coding a picture isupdated as follows:R=R−S _(i,p,b)

S_(i,p,b) is the coded bits for the current frame.

-   Before coding a GOP, the remaining bits for the current GOP is    initialized as follows:    R=G+R _(prev)    G=bit_rate×N÷picture_rate-   here, R is the remained bits for the current GOP.-   N is the number of frames in current GOP.-   G is the number of bits for a GOP.-   R_(prev) is the remained bits for the previous GOP. For the first    GOP, R_(prev)=0.

X_(i), X_(p) and X_(b) are initialized as:X _(i) =a×bit_rateX _(p) =b×bit_rateX _(b) =c×bit_rate

here a, b and c are constants.

bit_rate is the target bitrate.

Does the mode selection while using the quantization parameter ofprevious macroblock as a prediction value for the current macroblock.The mode minimizes the following expression is selected as the initialcoding mode for the current macroblock:D(s,c,MODE|QP)+λ_(MODE)R(s,c,MODE|QP)

here, s is the luma value of the original macroblock. c is the lumavalue of the reconstructed macroblock. λ_(MODE) is the lagrangianconstant.

For I/P frame, λ_(MODE)=0.85×2^(Q) ^(m-1) ^(/3);

For B frame, λ_(MODE)=4×0.85×2^(Q) ^(m-1) ^(/3).

D(s,c,MODE|QP) is used to evaluate the distortion of the currentmacroblock after it is coded.

R(s,c,MODE|QP) is the bits used to code the macroblock with mode MODE.

QP is the quantization parameter for current macroblock. for motionestimation in P or B frame, the motion vectors minimizes followingexpression are selected as the motion vectors for the currentmacroblock:J(m,λ _(MOTION))=SA(T)D(s,c(m))+λ_(MOTION) R(m−p)

-   here, D(s,c(m)) is used to evaluate the distortion from motion    compensation.-   SA (T) D is sum of the absolute difference (or after Hadmard    transform) for the macroblock.-   R(m−p) is the bits used to code the motion vector.-   s is the luma value of the current macroblock in the original frame.-   c is the luma value in reference picture.-   m is the motion vector.-   p is the predicted motion vector.-   λ_(MOTION) is the lagrangian constant and λ_(MOTION)=√{square root    over (λ_(MODE))}.-   λ_(MODE) is the lagrangian constant.

After the first rate distortion mode selection, the output of RDO modeselection module is sent to JVT processing module. A new quantizationparameter will be calculated by the JVT processing module. Thequantization parameter is adjusted according to macroblock activity.

After first rate distortion mode selection, the sum of the absolutedifference is used as the macroblock activity estimation. The macroblockactivity is calculated as:

${act}_{m} = {{\sum\limits_{i,j}{{{{s\left( {i,j} \right)} - {c\left( {i,j} \right)}}}\mspace{14mu}{N\_{ac}t}_{m}}} = \frac{\left( {2 \times {act}_{j}} \right) + {avg\_ act}}{{act}_{j} + \left( {2 \times {avg\_ act}} \right)}}$here, i is the horizontal position of the pixel in the currentmacroblock. j is the vertical position of the pixel in the currentmacroblock. N_act_(m) is the activity of the current macroblock. s(i,j)is the luma value of the original pixel(i,j), c(i,j) is the predictionvalue of pixel(i,j). avg_act is the average act, in the previous codedpicture which is coded with the same type as current picture. act_(m) isthe sum of the absolute difference after motion compensation or intraprediction.

When coding the first frame, the virtual buffer occupancy is initializedwith:d ₀ ^(i)=10×r/31d ₀ ^(P) =K _(p) ×d ₀ ^(i)d ₀ ^(b) =K _(b) ×d ₀ ^(i)

here r is the virtual buffer size; d₀ ^(i), d₀ ^(p), and d₀ ^(b) is theinitial virtual buffer occupancy for i, p, or b frame. K_(p) is thecomplexity ration between I, P frame; K_(b) is the complexity ratiobetween I, B frame.

The RDO based rate control also includes a second RDO mode selection,after quantization parameter decision for the current macroblock. Thatis to say, the decided quantization parameter for the current macroblockwill be used to RDO mode selection again. The mode which minimizes thefollowing expression will be selected as coding mode for the currentmacroblock:

here, s is the luma value of the original macroblock. c is the lumavalue of the reconstructed macroblock. λ_(MODE) is the lagrangianconstant.

For I/P frame, λ_(MODE)=0.85×2^(Q) ^(m-1) ^(/3);

For B frame, λ_(MODE)=4×0.85×2^(Q) ^(m-1) ^(/3).

D(s,c,MODE|QP) is used to evaluate the distortion of the currentmacroblock after it is coded.

R(s,c,MODE|QP) is the bits used to code the macroblock with mode MODE.

QP is the quantization parameter for current macroblock.

Quantization parameter from JVT processing module is sent back to JVTprocessing module, the macroblock is coded by JVT processing module andoutput.

Based on above modules, the drawbacks of traditional rate controlschemes are removed. As RDO and rate control are considered together,the RDO based video coding can reach accurate target bitrate controlwhile with good performance.

1. A method for rate distortion optimization (RDO) based rate controlcomprising: Step 1: performing bit allocation for every picture in a GOPwhich includes an I frame, a P frame, or a B frame, and based on theallocated bits a predicted quantization parameter being used to do afirst rate distortion optimization mode selection for every macroblockin the current picture, wherein the predicted quantization parameter isthe quantization parameter of a previous macroblock, and a coding modeminimizing the following formula is selected as the initial coding modefor the current macroblock:D(s,c,MODE|QP)+λ_(MODE)R(s,c,MODE|QP) wherein s is the luma value of theoriginal macroblock, c is the luma value of the reconstructedmacroblock, λ_(MODE) is the lagrangian constant; for I/P frame,λ_(MODE)=0.85×2^(Q) ^(m-1) ^(/3); for B frame, λ_(MODE)=4×0.85×2^(Q)^(m-1) ^(/3); D(s, c,MODE|QP) is used to evaluate the distortion of thecurrent macroblock after it is coded with mode MODE; R(s,c,MODE|QP) isthe bits used to code the macroblock with mode MODE; QP is aquantization parameter for the current macroblock, and is equal to thepredicted quantization parameter in the first rate distortion modeselection; Step 2: the information collected from the first ratedistortion mode selection being used to calculate a final quantizationparameter for rate control, and if the final quantization parameter isdifferent from the predicted, a second rate distortion mode selectionwill be executed again.
 2. The method of claim 1, wherein in step 1,before coding a GOP, bit allocation for the pictures in the GOP with theaverage picture size is performed.
 3. The method of claim 2, wherein theaverage picture size is calculated as: R/F=R÷F, here, R is the targetbit rate. F is the picture rate. R/F is the average picture size.
 4. Themethod of claim 1, further comprising performing bit allocationadjustment in the coded GOP, wherein the adjustment is implemented asfollows:${T_{i} = {\max\left\{ {\frac{R}{1 + \frac{N_{p}X_{p}}{K_{p}X_{i}} + \frac{N_{b}X_{b}}{K_{b}X_{i}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}}\;$$T_{p} = {\max\left\{ {\frac{R}{N_{p} + \frac{N_{b}K_{p}X_{b}}{K_{b}X_{p}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}$$T_{b} = {\max\left\{ {\frac{R}{N_{p} + \frac{N_{p}K_{b}X_{p}}{K_{p}X_{b}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}$here, T_(I), T_(p) and T_(b) is the bits allocated to the I, P or Bframe respectively; N_(i), N_(p) and N_(b) is the remained none coded I,P or B frames in the GOP respectively; X_(i), X_(p) and X_(b) is theglobal complexity estimation for the I, P or B frame respectively and isdefined as the multiplier between coded bits and average quantizationparameter for the frame; bit_rate is the target bit rate, picture_rateis the frame rate; K_(p) and K_(b) are constants, K_(p), K_(b) means thecomplexity ratio between P, B frame and I frame respectively; R is theremained bits for the GOP, and after coding a picture it is updated asfollows:R=R−S _(i,p,b) S_(i,p,b) is the coded bits for the current frame.
 5. Themethod of claim 4, wherein before coding a GOP, the remaining bits forthe current GOP is initialized as follows:R=G+R _(prev)G=bit_rate×N÷picture_rate here, R is the remained bits for the currentGOP; N is the number of frames in the current GOP; G is the number ofbits for a GOP; R_(prev) is the remained bits for the previous GOP, forthe first GOP, R_(prev)=0.
 6. The method of claim 4, wherein X_(i),X_(p) and X_(b) are initialized as:X _(i) =a×bit_rateX _(p) =b×bit_rateX _(b) =c×bit_rate here a, b and c are constants; bit_rate is the targetbitrate.
 7. The method of claim 1, wherein for motion estimation in P orB frame, the motion vector minimizing following expression is selectedas the motion vector for the current macroblock:J(m,λ _(MOTION))=SA(T)D(s,c(m))+λ_(MOTION) R(m−p) here, D(s,c(m)) isused to evaluate the distortion from motion compensation; SA(T)D is thesum of the absolute difference after prediction (or after Hadmardtransform) for the macroblock; R(m−p) is the bits used to code themotion vector; s is the luma value of the current macroblock in theoriginal frame; c is the luma value in reference picture; m is themotion vector; p is the predicted motion vector; λ_(MOTION) is thelagrangian constant and λ_(MOTION)=√{square root over (λMODE)}; λ_(MODE)is the lagrangian constant.
 8. The method of claim 2, wherein after thefirst rate distortion mode selection, the RDO based rate control furtherincludes: calculating quantization parameter for the current macroblock;the quantization parameter is adjusted according to the macroblockactivity and buffer status.
 9. The method of claim 8, wherein thequantization parameter for the macroblock is adjusted according to themacroblock activity; after the first rate distortion mode selection, thesum of the absolute difference is used as the macroblock activityestimation; the macroblock activity is calculated as:${act}_{m} = {{\sum\limits_{i,j}{{{{s\left( {i,j} \right)} - {c\left( {i,j} \right)}}}\mspace{14mu}{N\_{ac}t}_{m}}} = \frac{\left( {2 \times {act}_{j}} \right) + {avg\_ act}}{{act}_{j} + \left( {2 \times {avg\_ act}} \right)}}$here, i is the horizontal position of the pixel in the currentmacroblock; j is the vertical position of the pixel in the currentmacroblock; N_act_(m) is the activity of the current macroblock; s(i,j)is the luma value of the original pixel(i, j), c(i, j) is the predictionvalue of pixel(i, j); avg_act is the average act_(m) in the previouscoded picture which is coded with the same type as current picture;act_(m) is the sum of the absolute difference after motion compensationor intra prediction.
 10. The method of claim 8, wherein a virtual bufferis used to do rate control; first set up the mapping from the virtualbuffer occupancy to macroblock quantization parameter, and the finalmacroblock quantization parameter is calculated as:$Q_{m} = {\left( \frac{d_{m}^{n} \times 31}{r} \right) \times {N\_{ac}t}_{m}}$d_(m)^(n) = d₀^(n) + B_(m − 1) − T_(n) × (m − 1)/MB_CNTr = 2 × bit_rate/picture_rate here, Q_(m) is the quantization parameterof current macroblock; d_(m) ^(n) is the current buffer occupancy, andit equals d_(m) ^(I), d_(m) ^(p), and d_(m) ^(b) for I, P, B framerespectively; B_(m-1) is the bits used to code previous macroblock; d₀^(n) is the initial buffer occupancy for current frame, n is i, p or b,corresponding to d₀ ^(i),d₀ ^(p),and d₀ ^(b); r is the size of virtualbuffer.
 11. The method of claim 10, wherein, when coding the firstframe, the virtual buffer occupancy is initialized with:d ₀ ^(b) =K _(b) ×d ₀ ^(i)d ₀ ^(i)=10×r/31d ₀ ^(p) =K _(p) d× ₀ ^(i) here r is the virtual buffer size; d₀ ^(i),d₀^(p), and d₀ ^(b) is the initial virtual buffer occupancy for i, p, or bframe; K_(p) is the complexity ratio between I, P frame; K_(b) is thecomplexity ratio between I, B frame.
 12. The method of claim 11, whereinthe RDO based rate control also includes a second RDO mode selection,after calculating the final quantization parameter for the currentmacroblock; thus the selected quantization parameter for the currentmacroblock will be used to do RDO mode selection again; the mode whichminimizes the following expression will be selected as the coding modefor the current macroblock:D(s,c,MODE|QP)+λ_(MODE)R(s,c,MODE|QP) here, s is the luma value of theoriginal macroblock, c is the luma value of the reconstructedmacroblock, λ_(MODE) is the lagrangian constant; for I/P frame,λ_(MODE)=0.85×2^(Q) ^(m-1) ^(/3); for B frame, λ_(MODE)=4×0.85×2^(Q)^(m-1) ^(/3); D(s,c,MODE|QP) is used to evaluate the distortion of thecurrent macroblock coded with mode MODE; R(s,c,MODE|QP) is the bits usedto code the macroblock with mode MODE; QP is the quantization parameterfor the current macroblock.
 13. The method of claim 12, wherein formotion estimation in P or B frame, the motion vector minimizingfollowing expression is selected for the current macroblock:J(m,λ _(MOTION))=SA(T)D(s,c(m))+λ_(MOTION) R(m−p) here, D(s,c(m)) isused to evaluate the distortion from motion compensation; SA(T)D is thesum of the absolute difference (or after Hadmard transform) for themacroblock; R(m−p) is the bits used to code the motion vector; s is theluma value of the current macroblock in the original frame; c is theluma value in reference picture; m is the motion vector; p is thepredicted motion vector; λ_(MOTION) is the lagrangian constant andλ_(MOTION)=√{square root over (λ_(MODE))}; λ_(MODE) is the lagrangianconstant.
 14. An apparatus for a rate distortion optimization (RDO)based rate control comprising: a video coding encoder module comprisinga JVT processing module, a RDO mode selection and adaptive quantizationmodule, a virtual buffer, and a global complexity estimation module;wherein, the JVT processing module receives an input frame, and isconnected with the RDO mode selection and adaptive quantization module,the virtual buffer and the global complexity estimation module; the RDOmode selection and adaptive quantization module is connected with thevirtual buffer and the global complexity estimation module, and isconfigured to receive an input signal from the JVT processing module,and to process the input signal based on the status of the virtualbuffer and the global complexity module; bit allocation for everypicture in a GOP which includes an I frame, a P frame, or a B frame isconducted, a first rate distortion optimization mode selection for everymacroblock in the current picture using a predicted quantizationparameter based on the allocated bits is conducted, wherein thepredicted quantization parameter is the quantization parameter of aprevious macroblock, and a coding mode minimizing the following formulaas the initial coding mode for the current macroblock is selected:D(s,c,MODE|QP)+λ_(MODE)R(s,c,MODE|QP) where s is the luma value of theoriginal macroblock, c is the luma value of the reconstructedmacroblock, λ_(MODE) is the lagrangian constant; for I/P frame,λ_(MODE)=0.85×2^(Q) ^(m-1) ^(/3); for B frame, λ_(MODE)=4×0.85×2^(Q)^(m-1) ^(/3); D(s,c,MODE|QP) is used to evaluate the distortion of thecurrent macroblock after it is coded with mode MODE; R(s,c,MODE|QP) isthe bits used to code the macroblock with mode MODE; QP is thequantization parameter for the current macroblock, and is equal to thepredicted quantization parameter in the first rate distortion modeselection; and the RDO mode selection and adaptive quantization moduleis configured to calculate a final quantization parameter for ratecontrol using the information collected from the first rate distortionmode selection, and if the final quantization parameter is differentfrom the predicted, execute a second rate distortion mode selection; theJVT processing module is configured to output the final coded macroblockwith the calculated parameter.
 15. The apparatus of claim 14, whereinbefore coding a GOP, bit allocation for the pictures in the GOP with theaverage picture size is conducted.
 16. The apparatus of claim 15,wherein the average picture size is calculated as: R/F=R÷F, here, R isthe target bit rate, F is the picture rate, R/F is the average picturesize.
 17. The apparatus of claim 16, wherein a bit allocation adjustmentin the GOP is conducted, the adjustment is shown as follows:$\begin{matrix}{T_{i} = {\max\;\left\{ {\frac{R}{1 + \frac{{N_{p}}_{\;}X_{p}}{K_{p}X_{i}} + \frac{N_{b}X_{b}}{K_{b}X_{i}}},\frac{{bit}\text{\_}{rate}}{8 \times {picture\_ rate}}} \right\}}} \\{T_{b} = {\max\;\left\{ {\frac{R}{N_{b} + \frac{N_{p}K_{b}{Xp}}{K_{p}X_{b}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}} \\{T_{p} = {\max\;\left\{ {\frac{R}{N_{p} + \frac{N_{b}K_{p}X_{b}}{K_{b}X_{p}}},\frac{bit\_ rate}{8 \times {picture\_ rate}}} \right\}}}\end{matrix}$ here, T_(I), T_(p) and T_(b) is the bits allocated the tothe I, P or B frame respectively; N_(i), N_(p) and N_(b) is the remainednone coded I, P or B frames in the GOP respectively; X_(i), X_(p) andX_(b) is the global complexity estimation for the I, P or B framerespectively and is defined as the multiplier between the coded bits andaverage quantization parameter for the frame; bit_rate is the target bitrate, picture_rate is the frame rate; K_(p) and K_(b) are constants,K_(p), K_(b) means the complexity ratio between P, B frame and I framerespectively; R is the remained bits for the GOP, and after coding apicture it is updated as follows:R=R−S _(i,p,b) S_(i,p,b) is the coded bits for the current frame. 18.The apparatus of claim 17, wherein before coding a GOP, the remainingbits for the current GOP is initialized as follows:R=G+R _(prev)G=bit_rate×N÷picture_rate here, R is the remained bits for the currentGOP; N is the number of frames in current GOP; G is the number of bitsfor a GOP; R_(prev) is the remained bits for the previous GOP, for thefirst GOP, R_(prev)=0.
 19. The apparatus of claim 17, wherein X_(i),X_(p) and X_(b) are initialized as:X _(i) =a×bit_rateX _(p) =b×bit_rateX _(b) =c×bit_rate here a, b and c are constants; bit_rate is the targetbitrate.
 20. The apparatus of claim 14, wherein for motion estimation inP or B frame, the motion vector minimizing following expression isselected for the current macroblock:J(m,λ _(MOTION))=SA(T)D(s,c(m))+λ_(MOTION) R(m−p) here, D(s,c(m)) isused to evaluate the distortion from motion compensation; SA(T)D is sumof the absolute difference (or after Hadmard transform) for themacroblock; R(m−p) is the bits used to code the motion vector; s is theluma value of the current macroblock in the original frame; c is theluma value in reference picture; m is the motion vector; p is thepredicted motion vector; λ_(MOTION) is the lagrangian constant andλ_(MOTION)=√{square root over (λ_(MODE))}; λ_(MODE) is the lagrangianconstant.
 21. The apparatus of claim 20, wherein after the first ratedistortion mode selection, the rate control scheme further includes:calculating a new quantization parameter and adjusting the newquantization parameter according to the macroblock activity and bufferstatus.
 22. The apparatus of claim 20, wherein for adjustingquantization parameter for the current macroblock, the sum of theabsolute difference is used as the macroblock activity estimation afterfirst rate distortion mode selection, and the macroblock activity iscalculated as:${act}_{m} = {{\sum\limits_{i,j}{{{{s\left( {i,j} \right)} - {c\left( {i,j} \right)}}}\mspace{14mu}{N\_{ac}t}_{m}}} = \frac{\left( {2 \times {act}_{j}} \right) + {avg\_ act}}{{act}_{j} + \left( {2 \times {avg\_ act}} \right)}}$here, i is the horizontal position of the pixel in the currentmacroblock, j is the vertical position of the pixel in the currentmacroblock, N_act_(m) is the activity of the current macroblock, s(i,j)is the luma value of the original pixel(i, j), c(i, j) is the predictionvalue of pixel(i, j), avg_act is the average act_(m) in the previouscoded picture which is coded with the same type as current picture,act_(m) is the sum of the absolute difference after motion compensationor intra prediction.
 23. The apparatus of claim 20, wherein the virtualbuffer is used to do rate control: first setting up the mapping from thevirtual buffer occupancy to macroblock quantization parameter, themacroblock quantization parameter is calculated as:$Q_{m} = {\left( \frac{d_{m}^{n} \times 31}{r} \right) \times {N\_{ac}t}_{m}}$r = 2 × bit_rate/picture_rated_(m)^(n) = d₀^(n) + B_(m − 1) − T_(n) × (m − 1)/MB_CNT here, Q_(m) isthe quantization parameter of current macroblock; d_(m) ^(n) is thecurrent buffer occupancy, and it equals d_(m) ^(I), d_(m) ^(p), andd_(m) ^(b) for I, P, B frame respectively; B_(m-1) is the bits used tocode previous macroblock; d₀ ^(n) is the initial buffer occupancy forcurrent frame, n is i, p or b, corresponding to d₀ ^(i),d₀ ^(p), and d₀^(b); r is the size of virtual buffer occupancy.
 24. The apparatus ofclaim 23, wherein when coding the first frame, the virtual bufferoccupancy is initialized with:d ₀ ^(i)=10×r/31d ₀ ^(p) =K _(p) ×d ₀ ^(i)d ₀ ^(b) =K _(b) ×d ₀ ^(i) here r is the virtual buffer size; d₀ ^(i),d₀^(p), and d₀ ^(b) is the initial virtual buffer occupancy for i, p, or bframe; K_(p) is the complexity ratio between I, P frame; K_(b) is thecomplexity ratio between I, B frame.
 25. The apparatus of claim 21,wherein the RDO based rate control also includes a second RDO modeselection, after quantization parameter decision for the currentmacroblock, thus the decided quantization parameter for the currentmacroblock will be used to do RDO mode selection again; the mode whichminimizes the following expression will be selected as coding mode forthe current macroblock:D(s,c,MODE|QP)+λ_(MODE)R(s,c,MODE|QP) here, s is the luma value of theoriginal macroblock, c is the luma value of the reconstructedmacroblock, λ_(MODE) is the lagrangian constant; for I/P frame,λ_(MODE)=0.85×2^(Q) ^(m-1) ^(/3); for B frame, λ_(MODE)=4×0.85×2^(Q)^(m-1) ^(/3); D(s,c,MODE|QP) is used to evaluate the distortion of thecurrent macroblock after it is coded; R(s,c,MODE|QP) is the bits used tocode the macroblock with mode MODE; QP is the quantization parameter forcurrent macroblock.
 26. The apparatus of claim 25, wherein for motionestimation in P or B frame, the motion vectors minimizes followingexpression are selected as the motion vectors for the currentmacroblock:J(m,λ _(MOTION))=SA(T)D(s,c(m))+λ_(MOTION) R(m−p) here, D(s,c(m)) isused to evaluate the distortion from motion compensation; SA(T)D is sumof the absolute difference (or after Hadmard transform) for themacroblock; R(m−p) is the bits used to code the motion vector; s is theluma value of the current macroblock in the original frame; c is theluma value in reference picture; m is the motion vector; p is thepredicted motion vector; λ_(MOTION) is the lagrangian constant andλ_(MOTION)=√{square root over (λ_(MODE))}; λ_(MODE) is the lagrangianconstant.
 27. The apparatus of claim 26, wherein quantization parameterfrom the RDO mode selection and adaptive quantization module is sentback to the JVT processing module, the macroblock is coded by the JVTprocessing module and output of the JVT processing module.